Crowdsourcing Formal Decision Making Using Generalized Semantic Games

We are after a Wikipedia for formal scientific knowledge; a crowdsourcing system where formal science claims are expressed as interpreted logical statements (a.k.a. claims). The crowd takes positions on claims and objectively argues these positions through Semantic Games (SGs). SGs are zero-sum, two-person games where players take two contradictory positions on claims and exchange examples and counter-examples to support their positions and dispute their opponent’s positions. SGs provide an attractive basis for solving some key challenges that face crowdsourcing systems. More concretely, 1) the challenge of defining user contributions is to some extent solved by SGs because SG players interact through a formal well-defined protocol. However, an SG is a binary interaction mechanism that needs to be scaled to the crowd. 2) SGs provide a basis for solving the challenges of evaluating users and their contributions because, under certain restrictions, SG winners are more likely to be stronger than their opponents, and the contributions of SG winners are more likely to be true than the contributions of their opponents. However, the system either has to guarantee those restrictions or somehow compensate for their absence. 3) SGs are fun to play, again under certain restrictions, and thus help address the user retention challenge. However, the challenge of combining user contributions is not addressed by SGs. The system should combine the results of several SGs to better evaluate users and their contributions. Our proposed system can be applied to crowdsource the decision of a formal science claim, the development of algorithms for solving formallyspecified computational problems as well as to educate and evaluate users. We developed a proof of concept implementation that employs firstorder logic to express claims and two algorithms to evaluate users and their contributions. We did not provide a particular Crowd Interaction Mechanism (CIM), instead we provided an architectural principle to organize CIMs that alleviate some of the concerns related to the meaningful evaluation of users. With the help of the committee members, we propose the following contributions: 1) to develop an SG-based Crowd Interaction Mechanism (CIM). 2) to generalize SGs, beyond arguments based on examples and counter-examples alone, to arguments that involve reductions (and other